AutoML Leaderboard

Best model	name	model_type	metric_type	metric_value	train_time
	1_Baseline	Baseline	logloss	0.660965	2.67
	2_DecisionTree	Decision Tree	logloss	0.640628	13.69
	3_Linear	Linear	logloss	0.132596	7.01
the best	4_Default_Xgboost	Xgboost	logloss	0.043949	15.64
	5_Default_NeuralNetwork	Neural Network	logloss	0.226807	3.69
	6_Default_RandomForest	Random Forest	logloss	0.102504	16.33
	Ensemble	Ensemble	logloss	0.043949	0.77

AutoML Performance

AutoML Performance Boxplot

Features Importance

Spearman Correlation of Models

Summary of 4_Default_Xgboost

<< Go back

Extreme Gradient Boosting (Xgboost)

• n_jobs: -1
• objective: binary:logistic
• eta: 0.075
• max_depth: 6
• min_child_weight: 1
• subsample: 1.0
• colsample_bytree: 1.0
• eval_metric: logloss
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

14.1 seconds

Metric details

	score	threshold
logloss	0.043949	nan
auc	0.998507	nan
f1	0.987342	0.569929
accuracy	0.990654	0.569929
precision	1	0.569929
recall	1	0.000103945
mcc	0.980134	0.569929

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.043949	nan
auc	0.998507	nan
f1	0.987342	0.569929
accuracy	0.990654	0.569929
precision	1	0.569929
recall	0.975	0.569929
mcc	0.980134	0.569929

Confusion matrix (at threshold=0.569929)

	Predicted as B	Predicted as M
Labeled as B	67	0
Labeled as M	1	39

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Decision plots

Top-10 Worst decisions for class 0 (Fold 1)

Top-10 Best decisions for class 0 (Fold 1)

Top-10 Worst decisions for class 1 (Fold 1)

Top-10 Best decisions for class 1 (Fold 1)

<< Go back

Summary of 3_Linear

<< Go back

Logistic Regression (Linear)

• n_jobs: -1
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

5.6 seconds

Metric details

	score	threshold
logloss	0.132596	nan
auc	0.991045	nan
f1	0.938272	0.372078
accuracy	0.953271	0.372078
precision	1	0.870131
recall	1	4.35226e-05
mcc	0.900865	0.372078

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.132596	nan
auc	0.991045	nan
f1	0.938272	0.372078
accuracy	0.953271	0.372078
precision	0.926829	0.372078
recall	0.95	0.372078
mcc	0.900865	0.372078

Confusion matrix (at threshold=0.372078)

	Predicted as B	Predicted as M
Labeled as B	64	3
Labeled as M	2	38

Learning curves

Coefficients

feature	Learner_1
texture_worst	2.08027
concavity_worst	1.83941
perimeter_worst	1.22598
radius_worst	1.17639
area_se	1.14835
compactness_worst	1.12662
area_worst	1.09382
perimeter_se	0.936844
symmetry_worst	0.756279
concavity_mean	0.65841
concave points_worst	0.585182
radius_se	0.484333
compactness_mean	0.35684
perimeter_mean	0.325049
smoothness_worst	0.320937
concave points_mean	0.299794
symmetry_mean	0.265584
smoothness_mean	0.188139
fractal_dimension_worst	0.166418
concavity_se	0.0808637
fractal_dimension_mean	0.0458945
symmetry_se	0.0424071
concave points_se	0.0245146
area_mean	0.0120686
smoothness_se	0.0111422
compactness_se	0.00852702
fractal_dimension_se	0.000435277
id	-0.105304
radius_mean	-0.150351
texture_mean	-0.640025
texture_se	-0.71434
intercept	-1.81986

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Decision plots

Top-10 Worst decisions for class 0 (Fold 1)

Top-10 Best decisions for class 0 (Fold 1)

Top-10 Worst decisions for class 1 (Fold 1)

Top-10 Best decisions for class 1 (Fold 1)

<< Go back

Summary of 2_DecisionTree

<< Go back

Decision Tree

• n_jobs: -1
• criterion: gini
• max_depth: 3
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

12.2 seconds

Metric details

	score	threshold
logloss	0.640628	nan
auc	0.914552	nan
f1	0.911392	0.514925
accuracy	0.934579	0.514925
precision	0.923077	0.514925
recall	0.975	0
mcc	0.859727	0.514925

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.640628	nan
auc	0.914552	nan
f1	0.911392	0.514925
accuracy	0.934579	0.514925
precision	0.923077	0.514925
recall	0.9	0.514925
mcc	0.859727	0.514925

Confusion matrix (at threshold=0.514925)

	Predicted as B	Predicted as M
Labeled as B	64	3
Labeled as M	4	36

Learning curves

Decision Tree

Tree #1

Rules

if (radius_worst <= 16.805) and (concave points_worst <= 0.152) and (perimeter_se <= 6.54) then class: B (proba: 97.01%) | based on 201 samples

if (radius_worst > 16.805) and (concavity_worst > 0.218) and (concavity_se > 0.018) then class: M (proba: 100.0%) | based on 98 samples

if (radius_worst <= 16.805) and (concave points_worst > 0.152) and (radius_mean <= 14.43) then class: M (proba: 100.0%) | based on 10 samples

if (radius_worst > 16.805) and (concavity_worst > 0.218) and (concavity_se <= 0.018) then class: M (proba: 66.67%) | based on 3 samples

if (radius_worst > 16.805) and (concavity_worst <= 0.218) and (radius_worst <= 18.705) then class: B (proba: 100.0%) | based on 3 samples

if (radius_worst > 16.805) and (concavity_worst <= 0.218) and (radius_worst > 18.705) then class: M (proba: 100.0%) | based on 2 samples

if (radius_worst <= 16.805) and (concave points_worst > 0.152) and (radius_mean > 14.43) then class: B (proba: 100.0%) | based on 1 samples

if (radius_worst <= 16.805) and (concave points_worst <= 0.152) and (perimeter_se > 6.54) then class: M (proba: 100.0%) | based on 1 samples

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Decision plots

Top-10 Worst decisions for class 0 (Fold 1)

Top-10 Best decisions for class 0 (Fold 1)

Top-10 Worst decisions for class 1 (Fold 1)

Top-10 Best decisions for class 1 (Fold 1)

<< Go back

Summary of 5_Default_NeuralNetwork

<< Go back

Neural Network

• n_jobs: -1
• dense_1_size: 32
• dense_2_size: 16
• learning_rate: 0.05
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

2.4 seconds

Metric details

	score	threshold
logloss	0.226807	nan
auc	0.978731	nan
f1	0.911392	0.82727
accuracy	0.934579	0.82727
precision	1	0.999511
recall	1	1.74313e-08
mcc	0.859727	0.82727

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.226807	nan
auc	0.978731	nan
f1	0.911392	0.82727
accuracy	0.934579	0.82727
precision	0.923077	0.82727
recall	0.9	0.82727
mcc	0.859727	0.82727

Confusion matrix (at threshold=0.82727)

	Predicted as B	Predicted as M
Labeled as B	64	3
Labeled as M	4	36

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 1_Baseline

<< Go back

Baseline Classifier (Baseline)

• n_jobs: -1
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

1.7 seconds

Metric details

	score	threshold
logloss	0.660965	nan
auc	0.5	nan
f1	0.544218	0.335737
accuracy	0.373832	0.335737
precision	0.373832	0.335737
recall	1	0.335737
mcc	0	0.335737

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.660965	nan
auc	0.5	nan
f1	0.544218	0.335737
accuracy	0.373832	0.335737
precision	0.373832	0.335737
recall	1	0.335737
mcc	0	0.335737

Confusion matrix (at threshold=0.335737)

	Predicted as B	Predicted as M
Labeled as B	0	67
Labeled as M	0	40

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of Ensemble

<< Go back

Ensemble structure

Model	Weight
4_Default_Xgboost	5

Metric details

	score	threshold
logloss	0.043949	nan
auc	0.998507	nan
f1	0.987342	0.569929
accuracy	0.990654	0.569929
precision	1	0.569929
recall	1	0.000103945
mcc	0.980134	0.569929

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.043949	nan
auc	0.998507	nan
f1	0.987342	0.569929
accuracy	0.990654	0.569929
precision	1	0.569929
recall	0.975	0.569929
mcc	0.980134	0.569929

Confusion matrix (at threshold=0.569929)

	Predicted as B	Predicted as M
Labeled as B	67	0
Labeled as M	1	39

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 6_Default_RandomForest

<< Go back

Random Forest

• n_jobs: -1
• criterion: gini
• max_features: 0.9
• min_samples_split: 30
• max_depth: 4
• eval_metric_name: logloss
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

14.8 seconds

Metric details

	score	threshold
logloss	0.102504	nan
auc	0.996269	nan
f1	0.962025	0.502086
accuracy	0.971963	0.636965
precision	1	0.636965
recall	1	0.00414039
mcc	0.940934	0.636965

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.102504	nan
auc	0.996269	nan
f1	0.961039	0.636965
accuracy	0.971963	0.636965
precision	1	0.636965
recall	0.925	0.636965
mcc	0.940934	0.636965

Confusion matrix (at threshold=0.636965)

	Predicted as B	Predicted as M
Labeled as B	67	0
Labeled as M	3	37

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Decision plots

Top-10 Worst decisions for class 0 (Fold 1)

Top-10 Best decisions for class 0 (Fold 1)

Top-10 Worst decisions for class 1 (Fold 1)

Top-10 Best decisions for class 1 (Fold 1)

<< Go back